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ABSTRACT 

Taking the position that science is not value-free 
has certain consequences. There are both ris:«s and rewards inherent 
when evaluators relinquish an objective, non-involved role for a more 
subjective, partisan role in an evaluation where issues of class, 
gender, and race are intertwined. It is important for evaluators to 
consider how their personal and political commitments to specific 
social programs affect their judgments of those programs, and how the 
interests of social justice can be served in promoting the interests 
of the clients for whom these programs are intended. The evaluators 1 
values helped to frame the evaluation of a teenage pregnancy 
prevention program in a rural southern community in the Florida 
panhandle. The evaluation approach was based on feminist concepts of 
how social research in general should be conducted and on the 
principles of action research. A high premium placed on interviewing 
program participants reflects a desire to serve the interests of 
social justice by giving a voice to people whom the decision makers 
rarely hear, and whose interests are rarely considered when decisions 
to terminate programs are made. Evaluators may face moral dilemmas in 
the future if the use of incentive programs with at-risk populations 
increases. Evaluators need to identify the moral and political stance 
they will assume in conducting evaluations in the future (MB) 
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Evaluation has traditionally been conceived of as an activity that involves the 
dispassionate judgment of an external reviewer who provides an assessment of the 
program's merit and worth. The underlying assumption of this position has been that 
human affairs can be subjected to the same standards of scientific rationality that 
characterizes practices in the natural sciences. Evahiators who violated this assumption by 
acknowledging their own values in relation to the program being evaluated ran the risk of 
being perceived as too subjectively involved to render an impartial judgment Values were 
seen as messy and confusing to the overall purpose of the evaluation, which was to 
determine (objectively, of course) the self evident "facts" of how the program operated, 
independent of the context in which it was applied. And as for any moral judgments about 
whether the program served the interests of social justice, or whether benefits were equally 
distributed, or whether program funds could better be applied elsewhere, these were issues 
no evahiator wished to ackno* /ledge, except on those rare occasions when conversations 
were "off the record, 11 and people were willing to admit their deep seated concerns. 

In this paper, my co-author and I take a very explicit stance toward declaring our 
values and describing how they frame the evaluation of a teen pregnancy prevention 
program. We share Guba and Lincoln's (1989) assertion that science is not value free, nor 
are evaluations in a fourth generation model. But such a position brings its own 
consequences. The purpose of this paper is to examine the risks and rewards inherent when 
evahiators relinquish an objective, non-involved role for a more subjective, partisan role in 
an evaluation where issues of class, gender, and race are intertwined. Specifically, my co- 
author and I are concerned with questions of how our personal and political commitments 
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to specific social programs affect our judgments of them, and how the interests of social 
justice can be served in promoting the interests of the clients for whom these programs are 
intended. 

Theoretical Perspectives 

Our position was developed from a number of theoretical perspectives. In a recent 
paper, Schwandt (1989) noted that evahiators should "recapture a moral discourse," a point 
of view shared by Ericson (1990), who suggested that evahiators have a moral obligation 
to consider the normative content of evaluation practice. These ideas are in line with the 
recent work of other theorists who have suggested that evaluation practices should reflect 
changes in philosophical orientations from a rationalist, utilitarian approach to a more 
humanistic, value laden approach (Guba & Lincoln, 1989; Lincoln & Guba, 1985; Stake, 
1986). A similar point of view has been espoused by Noddings (1984) and Soltis (1989) who 
argue that ethical considerations should frame the choice of educational and social research 
problems. Elsewhere, I have argued that social researchers have a responsibility to the 
larger public that cannot be evaded by simply appealing to abstract goals such as the 
pursuit of knowledge for its own sake (Emihovich, 1990). Those who conduct research 
within an evaluative context are obligated to consider the moral implications that their 
findings may carry for those who are affected by the program in question. Several minority 
re marchers have gone even further by suggesting that social science researchers should 
assume part of the responsibility for enhancing the quality of people's lives through their 
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work, a position that completely belies the concept of the disinterested scientist or 

evahiator. Our thinking was also influenced by feminist research, in that: 

Knowledge and politics are process rather than achievement, 
and that the commitment to engage in conversation to find out 
what the world is like is a moral and political commitment to 
a community...Through our decisions with a community, we 
decide how we want to belong to the world, how we want to 
set about understanding it, living in it, and changing it (Seller, 
1988, p. 180-181). 

Program Description 

The world we have been examining for the past tea months is located in a rural 

southern community in the Florida panhandle. The predominately minority county has the 

worst infant mortality rate in the state, and one of the highest teen pregnancy rates. The 

program we have been evaluating is operated by a community organization, and funded by 

a non-profit state agency. This community organization operates several programs to assist 

the youth of the community. One program in particular, called Brighter Futures, is for girls 

ages 14-17 who have already had one child, and who are enrolled with the intent of 

preventing a second pregnancy. The purpose of "Brighter Futures" is to develop peer 

support groups for these girls, based on a similar program known as the "Dollar-A-Day 

Teenage Pregnancy Prevention Program,'' which was developed by Dolgan and Goodman 

(1989). These support groups are created through the use of incentives in the form of 

weekly stipends ($7, plus $5 transportation for those girls who need to pay for rides), which 

are paid at the end of each meeting. A basic assumption is that while the girls may have 
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been motivated initially to join based on these stipends, additional incentives should arse 
from the peer support and counseling the girls receive from each other. The girls attend 
weekly meetings and plan social activities with the help of a trained counselor. A significant 
met about this program is that both the community agency and its programs are controlled / 
by African-American women. All of the participants enrolled in this program are also 
African-American women. 

The available literature on teen pregnancies suggests that successful prevention 
depends upon three factors: (1) adequate information about sexuality, reproduction, and 
family planning methods; (2) access to family planning services which are cost effective and 
easily obtained; and, (3) the teen mother's motivation to succeed, remain in school, and 
plan for a career (Moore, Simms, & Betsey, 1986). While the concept of "Brighter Futures" 
encompasses all three aspects, the primary focus is on developing the girls' self esteem and 
raising their aspirations for the future. According to Schorr & Schorr (1988), "The most 
fundamental reason for high rates of school-age pregnancy in the United States is that far 
too many youngsters reach adolescence without hopes or plans for a future that seem 
compelling enough to deter them from early parenthood" (p. 41). While an extensive 
literature on teen pregnancy is available, relatively few studies have examined the factors 
affecting the teen mothers' motivation and desire to change her life patterns. In addition, 
there have been no prior evaluation studies of incentive programs similar to this one. A 
substantive review of model programs for teenage pregnancy prevention does not mention 
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any incentive programs (Brindis & Jeremy, 1988). This evaluation was intended not only 
to provide feedback to program developers in terms of successful program implementation 
and achievement of program goals, it was also intended to contribute to the teen pregnancy 
literature in terms of identifying successful ways in which teen mothers can be motivated 
to change their behavior. 
Evaluation Approach 

The evaluation approach was based on feminist concepts of how social research in 
general should be conducted, following the work of Stacy (1988) and Seller (1988), and on 
the principles of action research. The agency director was very concerned that case studies 
be done to reflect the reality of the girls' lives, and to gain an in-depth understanding of 
the dynamics which lead to teenage pregnancy. In addition to compiling case studies, an 
ethnographic approach is being followed, with participant observations of Ae weekly 
meetings, interviews with all program staff, and the collection of all program documents. 
Notes from observations of related sites (e.g., executive board meetings, community 
gatherings, etc.) are also being kept Data from the girls* school and health records are also 
available. The program is funded until December, 1993. 

Because of the sensitive nature of the date being collected, the evaluation is very 
much oriented to the responsive, or stakeholder approach (Bryk, 1983). As Stecher and 
Davis (1987) defined it: 



Evahiators as moral agents 
AERA paper 1992 



Responsive evaluation is guided by the belief that the only 
meaningful evaluation is one that seeks to understand an issue 
from die multiple points of view of all people who have a stake 
in the program. The responsive evaluator does not believe that 
there is a single answer to a program question that can be 
found using tests, questionnaires, or statistical analyses. 
Instead, each person who is influenced by a program perceives 
it in a unique manner, and an evaluator can help try to answer 
program-related questions by portraying reality through the 
eyes of concerned constituents. The goal of a responsive 
evaluator is to facilitate efforts to understand a program from 
multiple perspectives (pp. 36-37). 

While this approach did not preclude the use of quantitative data (e.g., attendance rates 
at program meetings), a high premium was placed on interviewing these young women to 
learn how the program has made a difference in their lives, and in what ways it needed to 
be changed in order to help them. In effect, one purpose of this evaluation was to serve the 
interests of social justice by giving voice to people who the decision makers rarely hear, and 
whose interests are rarely considered when decisions to terminate programs are made 
(Sirotnik, 199^). 

In providing a means for these young women to voice their own concerns, a key 
component is the relationship of the evaluator to the people being evaluated. The 
relationships that we developed in the context of this work were of paramount concern, and 
led us to one of our questions: tc what extent do differences by class (both evahiators are 
university trained) and race (the first author is Caucasian, the second is African-American) 
affect our relationships with the participants? Are there co-occurring identities we all share 
(e.g., problems with being a mother) that can successfully bridge these perceived 
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differences? Recent literature on teen parents indicates that trust in terms of their 
willingness to share intimate, personal details of their lives is not built overnight, but 
instead over a period of several months, or even years (Dash, 1990). In our case, the feeling 
is that even after almost a year in this site, we are still relatively estranged from the girls, 
and that both class and racial differences are still problematic. 
Issues for discussion 

Given the length of time available in a presentation of this kind, it is impossible to 
provide the kind of rich data that led us to certain conclusions about the effectiveness of 
this program. Instead, we would like to share some of the concerns we have about doing 
this type of work, and how we are struggling to address them. 
Feminism and Program Development 

One issue concerns our feminist stance. As feminists, we are deeply committed to 
the type of services this program offers, We often ask ourselves to what extent do our 
commitments influence our ability to provide needed information to help improve the 
program? For example, we have already noted that this program is entirely controlled by 
African-American women. However, as outsiders, we see noticeable problems in the 
efectiveness of some of the counselors in working with the girls, a concern that is shared 
by the program director. Because women of color have rarely had the opportunity to 
manage their own programs, it is difficult to provide constructive feedback without 
appearing to overlay an agenda that is closer to our feminist concerns than it may be to this 
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organization's concerns. For example, we see the raising of the girls' self esteem as a critical 
issue of empowerment, of making them more conscious of their ability to take control of 
their own lives. To move the young women toward this goal, we feel there are specific 
activities that could occur within the group meetings. The question for us is how much do 
we promote this suggestion (through making programmatic recommendations) without 
becoming program developers? Right now the program director is seeking input for 
developing a curriculum guideline; how far should we go in sharing our ideas? Should we 
bring in literature or topics for discussion? The problem is complicated by the fact that 
these young women hold very negative impressions of feminism, a point already raised by 
bell hooks (1989). Yet we are disturbed by the idea that without knowledge of certain ideas 
that could lead them to reflect on their choices, these girls remain trapped in a cycle of 
poverty and despair. 
Reciprocity 

Another issue that concerns us is that of reciprocity. The question of reciprocity 
always arises in ethnographic research (or it should), yet the specifics of what is gained by 
the participants is rarely identified. For example, we would like to see an open discussion 
at AERA of the idea of sharing royalties from any publications with the people who 
contributed to it by sharing their lives, and a discussion of whether this practice should 
constitute one of the standards for qualitative research in general. But there is also the 
question of what counts as reciprocity, since the rewards for academicians are not 
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necessarily those desired by people outside our narrow community. I thought I had solved 
the reciprocity issue by telling the girls that we hoped to write a book, and that we would 
share the money received. The first question asked was, "How much will we get?" Not 
wanting to work under false pretenses, I said, "Probably not much. Academic books don't 
sell like Stephen King's books do." At that point, one girl asked me why I bothered to do 
it It was clear from their attitude that they couldn't see the point, and I couldn't think of 
an intelligent way to explain the system of rewards (publish or perish) that I operated 
under. We are learning that reciprocity can occur in small ways; driving a girl to the store 
to get diapers (there's no public transforation and many girls live way out in the country) 
is a welcome act. Class differences are magnified in small acts of charity; we take for 
granted the ability to simply get in the car and go. 

In their recent paper on standards for qualitative research, Howe and Eisenhart 
(1990) did not address the question of reciprocity. Even more troubling is that fact that we 
do not live in the community, nor do we share in the problems that exist in these girls lives. 
Stacy (1988) has already explored the question of whether there can be a feminist 
ethnography, and has argued that ethnographic research can put people at greater risk of 
exploitation, abandonment, and betrayal than positivistic research. These are strong words, 
bu the fact remains that the professional benefits of working in this community accrue to 
us, and it is not clear yet what advantages the girls gain by working with us. We are 
painfully aware of bell hook's comment that: 
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Even if perceived "authorities" writing about a group to which they do not 
belong and/or over which they wield power, are progressive, caring, and 
"right-on" in every way, as long as their authority is constituted by either the 
absence of the voices of the individuals whose experiences they seek to 
address, or the dismissal of the voices as unimportant, the subject-object 
dichotomy is maintained, and domination is enforced (1989, p. 43). 

Incentives for Participation 

One of the most critical issues concerns the fact that this program is incentive based, 
in that the participants are paid to attend and remain childless (or without a second child). 
These programs represent a growing movement of using incentives to promote desired 
social outcomes, the ethics of which have not yet been addressed. We have not yet thought 
through the implications of this practice, but we realize it as one freighted with moral 
consequences, especially since the line between individual choice and materia] needs can 
become very thin indeed, when one is dealing with a group as poor as this one is. 
Furthermore, in our state, the use of birth control implants is just beginning. What is the 
moral position of the evaluator in programs where these implants are advocated in subtly 
coercive ways? (We hasten to add this is not the case in this program, in fact it is quite the 
opposite. When Norplant was first made available through the local county public health 
unit, so many women requested it that the supply was exhausted in three days. However, 
I am aware of other programs in the state where the idea has been discussed to require 
women to use Norplant as a condition of receiving public benefits). 

Another unanticipated consequence of the girls using Norplant is that their risk of 
contracting AIDS has substantially increased. Before, the girls primarily relied upon 
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condom use, a method they didn't prefer because the men often refused to wear one, and 
often didn't have one available at the appropriate time. Since Norplant reduces the risks 
of becoming pregnant to virtually zero, the girls have increased their sexual activity. Given 
the alarming spread of AIDS in this county, a problem accentuated by the recent closing 
of the school based health clinic, as evahiators we could spend all our time and energy just 
urging the counselors to provide AIDS information in the group meetings. 
Political considerations 

This evaluation holds political ramifications in the sense that the programs 
emphasize the concept of African-American self-help, and stress the need for individual 
responsibility and personal choices, yet there are also structural social problems that need 
to be addressed if these girls are to be successful in later life. Two problems worth 
mentioning are the lack of public transportation to the state capital, where the job 
opportunities are greater, and the tracking problem in the local school system, where many 
of these girls are shunted into vocational tracks or into an alternative high school, despite 
test scores which reveal their potential for higher education. We see part of our role as 
helping the prog/am staff realize how a report detailing program success can be used 
politically to deny services in other critical areas. We must also muster energy to combat 
efforts by certain groups to use our reports to argue that all social problems in this 
community will be solved if these girls do not have a second child. The lack of appropriate 
child care services in this county is shameful, and with the current crisis in state funding, 
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all attention seems to be focused on reducing teen pregnancy, without considering what 
opportunities and services are available for those girls who already have one child and who 
choose not to become pregnant again. 
Conclusion 

As the paradigms for doing behavioral and social science research change, it is 
axiomatic that the researcher or evahiator's rcle will change as well While this shift is just 
now being addressed in the field, our position is that many different accounts by evaluators 
who want to incorporate a more explicitly normative stance in their woik need to be 
presented to keep the dialogue open, and to provide answers as to how these issues will be 
addressed. With respect to the evaluation literature, the voices of minority participants and 
program developers have been silent, a fact we hope to change by providing accounts of 
the girls* lives. We also hope to underscore the moral dilemmas evaluators may race in the 
future if the use of incentive programs with at-risk populations increases, and to suggest 
that evaluators need to identity the moral and political stance they will assume in 
conducting evaluations in the future. 
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